Surgery is the only viable treatment for cataract patients with visual acuity (VA) impairment. Clinically, to assess the necessity of cataract surgery, accurately predicting postoperative VA before surgery by analyzing multi-view optical coherence tomography (OCT) images is crucially needed. Unfortunately, due to complicated fundus conditions, determining postoperative VA remains difficult for medical experts. Deep learning methods for this problem were developed in recent years. Although effective, these methods still face several issues, such as not efficiently exploring potential relations between multi-view OCT images, neglecting the key role of clinical prior knowledge (e.g., preoperative VA value), and using only regression-based metrics which are lacking reference. In this paper, we propose a novel Cross-token Transformer Network (CTT-Net) for postoperative VA prediction by analyzing both the multi-view OCT images and preoperative VA. To effectively fuse multi-view features of OCT images, we develop cross-token attention that could restrict redundant/unnecessary attention flow. Further, we utilize the preoperative VA value to provide more information for postoperative VA prediction and facilitate fusion between views. Moreover, we design an auxiliary classification loss to improve model performance and assess VA recovery more sufficiently, avoiding the limitation by only using the regression metrics. To evaluate CTT-Net, we build a multi-view OCT image dataset collected from our collaborative hospital. A set of extensive experiments validate the effectiveness of our model compared to existing methods in various metrics. Code is available at: https://github.com/wjh892521292/Cataract OCT.
translated by 谷歌翻译
User-generated-content (UGC) videos have dominated the Internet during recent years. While many methods attempt to objectively assess the quality of these UGC videos, the mechanisms of human quality perception in the UGC-VQA problem is still yet to be explored. To better explain the quality perception mechanisms and learn more robust representations, we aim to disentangle the effects of aesthetic quality issues and technical quality issues risen by the complicated video generation processes in the UGC-VQA problem. To overcome the absence of respective supervisions during disentanglement, we propose the Limited View Biased Supervisions (LVBS) scheme where two separate evaluators are trained with decomposed views specifically designed for each issue. Composed of an Aesthetic Quality Evaluator (AQE) and a Technical Quality Evaluator (TQE) under the LVBS scheme, the proposed Disentangled Objective Video Quality Evaluator (DOVER) reach excellent performance (0.91 SRCC for KoNViD-1k, 0.89 SRCC for LSVQ, 0.88 SRCC for YouTube-UGC) in the UGC-VQA problem. More importantly, our blind subjective studies prove that the separate evaluators in DOVER can effectively match human perception on respective disentangled quality issues. Codes and demos are released in https://github.com/teowu/dover.
translated by 谷歌翻译
近年来,旨在在衣服变化下与人身份相匹配的换衣人重新识别(CC-REID)是近年来的一个新的研究主题。但是,典型的基于生物识别的CC-REID方法通常需要繁琐的姿势或身体部位估计器来从人类生物特征性状中学习布置性特征,这带有高计算成本。此外,由于监视图像的分辨率下降,性能受到了显着限制。为了解决上述限制,我们为CC-REID提出了一个有效的身份敏感知识传播框架(DECKPRO)。具体而言,引入了一个布 - 丝毫空间注意模块,以通过从人解析模块中获取知识来消除服装外观的注意力。为了减轻人类面孔的分辨率退化问题和对矿山身份敏感的提示,我们建议使用先前的面部知识恢复缺失的面部细节,然后将其传播到较小的网络。训练后,不再需要进行人类解析或面部修复的额外计算。广泛的实验表明,我们的框架的表现优于最先进的方法。我们的代码可在https://github.com/kimbingng/deskpro上找到。
translated by 谷歌翻译
组合多个传感器使机器人能够最大程度地提高其对环境的感知意识,并增强其对外部干扰的鲁棒性,对机器人导航至关重要。本文提出了可融合的基准测试,这是一个完整的多传感器数据集,具有多种移动机器人序列。本文提出了三项贡献。我们首先推进便携式和通用的多传感器套件,可提供丰富的感官测量值:10Hz激光镜点云,20Hz立体声框架图像,来自立体声事件相机的高速率和异步事件,来自IMU的200Hz惯性读数以及10Hz GPS信号。传感器已经在硬件中暂时同步。该设备轻巧,独立,并为移动机器人提供插件支持。其次,我们通过收集17个序列来构建数据集,该序列通过利用多个机器人平台进行数据收集来涵盖校园上各种环境。一些序列对现有的SLAM算法具有挑战性。第三,我们为将本地化和映射绩效评估提供了基础真理。我们还评估最新的大满贯方法并确定其局限性。该数据集将发布由原始传感器的设置,地面真相,校准数据和评估算法组成:https://ram-lab.com/file/site/site/multi-sensor-dataset。
translated by 谷歌翻译
当前的深度视频质量评估(VQA)方法通常在评估高分辨率视频时具有高计算成本。这使他们无法通过端到端培训学习更好的视频质量相关表示。现有方法通常考虑幼稚的采样以降低计算成本,例如调整大小和裁剪。但是,它们显然在视频中损坏了与质量相关的信息,因此并不是学习VQA的良好表示形式的最佳选择。因此,渴望为VQA设计一种新的质量保留抽样方案。在本文中,我们提出了网格迷你斑点采样(GMS),该采样允许通过在原始分辨率下采样贴片来考虑局部质量,并通过以统一网格采样的迷你绘制来涵盖全球质量。这些迷你斑点是剪接和对齐的,称为片段。我们进一步构建了专门设计的碎片注意网络(粉丝),以适应碎片作为输入。由片段和粉丝组成,VQA(快速VQA)提出的片段样品变压器可实现有效的端到端深VQA,并学习有效的与视频质量相关的表示。它可以提高最新准确性约10%,同时减少1080p高分辨率视频的99.5%的失败。新学习的与视频质量相关的表示形式也可以转移到较小的VQA数据集中,从而在这些情况下提高性能。广泛的实验表明,Fast-VQA在各种分辨率的输入方面具有良好的性能,同时保持高效率。我们在https://github.com/timothyhtimothy/fast-vqa上发布代码。
translated by 谷歌翻译
在现有作品中,框架及其对视频质量评估(VQA)的影响之间的时间关系仍然不足。这些关系导致视频质量的两种重要效果类型。首先,某些时间变化(例如摇动,闪烁和突然的场景过渡)会导致时间扭曲并导致额外的质量降解,而其他变化(例如,与有意义的事件相关的变化)却没有。其次,人类视觉系统通常对具有不同内容的框架有不同的关注,从而导致其对整体视频质量的重要性不同。基于变压器的突出时间序列建模能力,我们提出了一种新颖有效的基于变压器的VQA方法来解决这两个问题。为了更好地区分时间变化,从而捕获了时间变形,我们设计了一个基于变压器的时空扭曲提取(STDE)模块。为了解决时间质量的关注,我们提出了类似编码器的时间含量变压器(TCT)。我们还介绍了功能上的时间抽样,以减少TCT的输入长度,以提高该模块的学习效率和效率。由STDE和TCT组成,用于视频质量评估(DISCOVQA)的拟议的时间失真符合变压器(DISCOVQA)在几个VQA基准上达到了最新的性能,而无需任何额外的预训练数据集,多达10%的概括能力提高了10%比现有方法。我们还进行了广泛的消融实验,以证明我们提出的模型中每个部分的有效性,并提供可视化以证明所提出的模块实现了我们对这些时间问题进行建模的意图。我们将在以后发布我们的代码和预算权重。
translated by 谷歌翻译
联合学习在具有分布式数据的设备上进行模型,同时保护隐私并获取类似于集中式ML的模型。具有数据和计算能力的大量工人是联邦学习的基础。然而,不可避免的成本阻止自私的工人免费服务。此外,由于数据隔离,任务发布者缺乏选择,评估和支付具有高质量数据的可靠工人的有效方法。因此,我们设计了一种基于拍卖的激励机制,具有声誉和贡献测量的横向联合学习。通过设计合理的衡量贡献方法,我们建立了工人的声誉,这很容易下降,难以改善。通过反向拍卖,工人竞标任务,任务发布者选择合作者组合声誉和出价价格。通过预算制约,获奖工人根据业绩支付。我们证明我们的机制满足诚实的工人,预算可行性,真实性和计算效率的个人合理性。
translated by 谷歌翻译
今天的数据往往散布数十亿资源受限的边缘设备,具有安全性和隐私约束。联合学习(FL)已成为在保持数据私有的同时学习全球模型的可行解决方案,但FL的模型复杂性被边缘节点的计算资源阻碍。在这项工作中,我们调查了一种新的范例来利用强大的服务器模型来突破FL中的模型容量。通过选择性地从多个教师客户和本身学习,服务器模型开发深入的知识,并将其知识传输回客户端,以恢复它们各自的性能。我们所提出的框架在服务器和客户端模型上实现了卓越的性能,并在统一的框架中提供了几个优势,包括异构客户端架构的灵活性,对各种图像分类任务的客户端和服务器之间的通信效率。
translated by 谷歌翻译
Knowledge distillation is often used to transfer knowledge from a strong teacher model to a relatively weak student model. Traditional knowledge distillation methods include response-based methods and feature-based methods. Response-based methods are used the most widely but suffer from lower upper limit of model performance, while feature-based methods have constraints on the vocabularies and tokenizers. In this paper, we propose a tokenizer-free method liberal feature-based distillation (LEAD). LEAD aligns the distribution between teacher model and student model, which is effective, extendable, portable and has no requirements on vocabularies, tokenizer, or model architecture. Extensive experiments show the effectiveness of LEAD on several widely-used benchmarks, including MS MARCO Passage, TREC Passage 19, TREC Passage 20, MS MARCO Document, TREC Document 19 and TREC Document 20.
translated by 谷歌翻译
Graph Neural Networks (GNNs) have attracted increasing attention in recent years and have achieved excellent performance in semi-supervised node classification tasks. The success of most GNNs relies on one fundamental assumption, i.e., the original graph structure data is available. However, recent studies have shown that GNNs are vulnerable to the complex underlying structure of the graph, making it necessary to learn comprehensive and robust graph structures for downstream tasks, rather than relying only on the raw graph structure. In light of this, we seek to learn optimal graph structures for downstream tasks and propose a novel framework for semi-supervised classification. Specifically, based on the structural context information of graph and node representations, we encode the complex interactions in semantics and generate semantic graphs to preserve the global structure. Moreover, we develop a novel multi-measure attention layer to optimize the similarity rather than prescribing it a priori, so that the similarity can be adaptively evaluated by integrating measures. These graphs are fused and optimized together with GNN towards semi-supervised classification objective. Extensive experiments and ablation studies on six real-world datasets clearly demonstrate the effectiveness of our proposed model and the contribution of each component.
translated by 谷歌翻译